A MapReduce and MPI Programming Model for Distributed Large Scale 3D Mesh Processing
نویسندگان
چکیده
Developing a high performance platform for large-scale, high-intensity data processing is a priority for researching cost-effective parallel finite element methods (FEM). This paper introduces an efficient MapReduce-MPI based strategy for parallel 3D finite element mesh processing, demonstrates the potential benefits of this approach for optimally utilizing system resources. Preliminary experimental results show that the new platform improves speedup over a range of problem sizes and different machine numbers. In detail, this paper includes the design of scalable Hadoop algorithms for 3D FEM mesh processing; experimental evaluation of these algorithms on computer clusters; and discussions on the benefits and challenges of developing 3D FEM algorithms using the MapReduce-MPI model.
منابع مشابه
A Cloud Computing Platform for Large-Scale Forensic Computing
The timely processing of massive digital forensic collections demands the use of large-scale distributed computing resources and the flexibility to customize the processing performed on the collections. This paper describes MPI MapReduce (MMR), an open implementation of the MapReduce processing model that outperforms traditional forensic computing techniques. MMR provides linear scaling for CPU...
متن کاملAdaptive Dynamic Data Placement Algorithm for Hadoop in Heterogeneous Environments
Hadoop MapReduce framework is an important distributed processing model for large-scale data intensive applications. The current Hadoop and the existing Hadoop distributed file system’s rack-aware data placement strategy in MapReduce in the homogeneous Hadoop cluster assume that each node in a cluster has the same computing capacity and a same workload is assigned to each node. Default Hadoop d...
متن کاملSoftware Design and Implementation for MapReduce across Distributed Data Centers
Recently, the computational requirements for large-scale data-intensive analysis of scientific data have grown significantly. In High Energy Physics (HEP) for example, the Large Hadron Collider (LHC) produced 13 petabytes of data in 2010. This huge amount of data are processed on more than 140 computing centers distributed across 34 countries. The MapReduce paradigm has emerged as a highly succ...
متن کاملMapReduce in MPI for Large-scale graph algorithms
We describe a parallel library written with message-passing (MPI) calls that allows algorithms to be expressed in the MapReduce paradigm. This means the calling program does not need to include explicit parallel code, but instead provides “map” and “reduce” functions that operate independently on elements of a data set distributed across processors. The library performs needed data movement bet...
متن کاملLarge Scale Image Processing Using Distributed and Parallel Architecture
--Enormous amount of images are uploaded and used via internet and this ratio is still drastically increasing .So there is instant need to handle this data and customize and filter it as per user and application requirement. This paper describe some parallel and distributed processing techniques like Hadoop, HIPI, Map reduce, CUDA, MPI to process massive database. Mapreduce based large-scale im...
متن کامل